There are many situations in which it is required to manage a portfolio of experts such as automated problem solvers, algorithms, automated services or human experts in order to select the most appropriate expert for a given task. When faced with a diverse set of tasks, an approach which would be effective for all of those tasks is likely to be complex. It is therefore typically necessary to use a portfolio of experts which specialize on different types of task. However, there is then the problem of how to select the most appropriate expert for a given task from the portfolio of experts in an automated manner which is simple and effective.
The selection problem is further exacerbated since the nature of the tasks being solved may change with time and in addition, the performance of the experts may change.
An example class of tasks where an algorithm portfolio approach can be beneficial is solving hard combinatorial problems where typically, there does not exist one single approach that outperforms any other across a range of real-world problem instances. Because of their exponentially large search spaces, these problems are in general computationally intractable and depending on the properties of the problem one particular automated problem solver performs better than another according to some measure (e.g. runtime). A vast discrepancy in performance arises because solvers commit to different heuristic choices or employ different techniques. However it is difficult to produce an automated process for selecting the most appropriate automated problem solver to use from a portfolio of automated problem solvers given a particular task.
There is a need to enable expert selection systems to be updated on-the-fly such that expert performance feedback is taken into account as soon as possible. Where large numbers of experts and tasks are involved this problem is particularly acute.
The embodiments described below are not limited to implementations which solve any or all of the disadvantages of known automated expert selection systems.